Design and Evaluation of Smart Disk Architecture for Data-Intensive Applications

The requirements for storage space and computational power of large-scale applications are increasing rapidly. Clusters seem to be the most attractive architecture for such applications, due to their low costs and high scalability. On the other hand, smart disk systems, with their large storage capacities and growing computational power are becoming increasingly popular. In this work, we compare the performance of these architectures along with a single host based system by using representative queries from the Decision Support System (DSS) databases. We show how to implement individual database operations in the smart disk system and also show how to optimize the execution of the whole query by bundling frequently occurring individual operations together and executing the bundle in a single invocation. Then, we present a protocol we devised for minimizing the communication time in the smart disk based system. To measure the response times, we have developed the DBsim, an accurate simulator which can simulate the database operations for the single host based, cluster based and smart disk based systems. Using this simulator, we illustrate that the smart disk architecture brings substantial benefits in terms of overall query execution times. In particular, the average response time of smart disk architecture for the representative queries from the TPC-D benchmark in our base configuration is 71% smaller than the response time on the single host based system and 4:2% smaller than the response time on the fastest cluster architecture. Dept. of Computer Science and Engineering, The Pennsylvania State University, University Park, PA 16802.